BUG: Presence of softlink in HDF5 file breaks HDFStore.keys() (GH20523) #20537
Conversation
TomAugspurger
added
IO Data
| @@ -1073,10 +1073,11 @@ def groups(self): | |||
| self._check_if_open() | |||
| return [ | |||
| g for g in self._handle.walk_nodes() | |||
| if (getattr(g._v_attrs, 'pandas_type', None) or | |||
| getattr(g, 'table', None) or | |||
| if (not isinstance(g, _table_mod.link.Link) and | |||
There was a problem hiding this comment.
Are there any other cases we should exclude, or just Link?
There was a problem hiding this comment.
Link is the abstract base class for SoftLink and ExternalLink in pytables. These subclasses do not support attributes as of HDF5 (this appears to be a limitation of the HDF5 library but may change in the future, reference: http://www.pytables.org/usersguide/libref/link_classes.html). I believe all other node types support attributes. A quick search of NoAttrs in Pytables shows usage only in tables/link.py.
Codecov Report
@@ Coverage Diff @@
## master #20537 +/- ##
==========================================
+ Coverage 91.84% 91.84% +<.01%
==========================================
Files 152 152
Lines 49255 49255
==========================================
+ Hits 45238 45240 +2
+ Misses 4017 4015 -2
Continue to review full report at Codecov.
|
pandas/tests/io/test_pytables.py
Outdated
|
|
||
| with ensure_clean_store('tmp.__%s__.h5' % tm.rands(10)) as store: | ||
|
|
||
| # GH 20523 |
There was a problem hiding this comment.
can you move this test right after test_keys (in the class)
pandas/tests/io/test_pytables.py
Outdated
|
|
||
| def test_ignore_hdf_softlink(): | ||
|
|
||
| with ensure_clean_store('tmp.__%s__.h5' % tm.rands(10)) as store: |
There was a problem hiding this comment.
move the comment right below the test title
doc/source/whatsnew/v0.23.0.txt
Outdated
| @@ -1020,6 +1020,7 @@ I/O | |||
| - Bug in :meth:`pandas.io.stata.StataReader.value_labels` raising an ``AttributeError`` when called on very old files. Now returns an empty dict (:issue:`19417`) | |||
| - Bug in :func:`read_pickle` when unpickling objects with :class:`TimedeltaIndex` or :class:`Float64Index` created with pandas prior to version 0.20 (:issue:`19939`) | |||
| - Bug in :meth:`pandas.io.json.json_normalize` where subrecords are not properly normalized if any subrecords values are NoneType (:issue:`20030`) | |||
| - Bug in :func:`HDFStore.groups` when reading a file with a softlink causes exception (:issue:`20523`) | |||
There was a problem hiding this comment.
say .keys instead here as that's more user facing.
dworvos
force-pushed
the
fix-softlink-hdf5
branch
from
8112132 to
80033c7
Compare
dworvos
force-pushed
the
fix-softlink-hdf5
branch
from
80033c7 to
e0a21d6
Compare
|
Hi, had some newbie git issues yesterday with rebasing, but I think I resolved them now in my branch/fork - please let me know if there are outstanding issues. |
|
thanks @dworvos |
git diff upstream/master -u -- "*.py" | flake8 --diff